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M,  A.  Hargolis 

The  RAND  Corporation,  Santa  Monica,  California 

INTRODUCTION 

I.  Subject  of  the  Briefing 

A.  Resource  analysis  role  in  cost-effectiveness  analysis;  the 
nature  of  resource  analysis,  the  analytical  tools  it  employs, 
and  its  relationship  with  the  effectiveness  side  of  the 
equation. 

B.  Limitation  to  the  aerospace  industry,  DOD,  and  NASA 

1,  This  industry  and  the  government  organizations  it 
serves  have  been  the  breeding  ground  of  cost- 
effectiveness  analysis  because  of  their  uniquely 
complicated  planning  problems. 

2.  Experience  of  the  speaker,  and  probably  most  of 
the  audience. 

II.  Varieties  of  Resource  Analysis 

A.  Emphasis  of  the  briefing  on  enumerating  and  distinguishing 
among  the  varieties  of  resource  analysis  commonly  required 
in  cost-effectiveness  analysis. 

B.  Variations  due  to  differing  levels  of  aggregation 

1.  Force  structure/total  plan 

2.  Individual  weapon  system  or  space  project 

3.  Individual  equipment  or  operation 

•k 

Any  views  expressed  in  this  paper  are  those  of  the  author. 

They  should  not  be  interpreted  as  reflecting  the  views  of  The  RAND 
Corporation  or  the  official  opinion  of  policy  of  any  of  its  govern¬ 
mental  or  private  research  sponsors.  Papers  are  reproduced  by  The 
RAND  Corporation  aa  a  courtesy  to  members  of  its  staff. 

This  paper  is  to  be  given  on  April  13,  1966,  as  part  of  a  lecture 
series  sponsored  by  the  American  Institute  of  Aeronautics  and 
Astronautics. 
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C,  Variations  due  to  differing  time  contexts 

X.  Long  range 

2.  Short  term 

VARIETIES  CT  RESOURCE  ANALYSIS 

I.  Individual  Equipment  or  Operation 

II.  Individual  Weapon  System  Costing 

A.  Input  variables 

1.  H5E  and  AGE  description 

a.  design  and  performance  data 

2.  Operational  and  organizational  description 

3.  banning  policy 

4.  In  commission  rate  data 

a.  alert  status 

5.  Maintenance  concept 

5.  Training  data 

7.  Ballistic  missile  illustration  charts,  more  detailed 
presentation 

B.  Cost  element  breakout/work  breakdown  structure 

1.  Appropriate  choice  not  a  matter  of  maximum  detail 
but  availability  of  information  to  the  decision¬ 
maker  (or  his  cost  analyst)  at  the  point  in  time 
the  decision  must  be  made. 

2.  Categories  should  be  structured  to  be  of  maximum 
use  in  the  analytical  problem  at  hand;  if  possible, 
they  should  be  used  to  highlight  the  differences 
among  the  alternatives  under  consideration — more 
aggregation  where  the  alternatives  are  alike,  less 
where  they  display  different  features. 

3.  Citing  a  cost  element  explicitly  rather  than  in  an 
aggregate  is  a  matter  both  of  its  relative  size  and 
variability  in  the  analysis  at  hand. 
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C.  Cost  categories  and  their  relative  time  impact 

1.  R&D 

2.  Investment 

3.  Annual  operating 

III.  Force  Structure  Costing 

A.  Definition:  determination  of  the  resource  impact  of  alter¬ 
native  future  force  proposals  (plans):  i.e.,  aggregation 

of  systems  (projects)  as  well  as  nonsystem-oriented  activities. 

1.  Nonsystem-oriented  activities:  Air  Training  Command 
or  Air  Force  Logistics  Command;  in  the  case  of  NASA, 
the  Office  of  Advanced  Research  and  Technology  (CART) 
or  the  Office  of  Tracing  and  Data  Acquisition  (OTDA). 

B.  Force  structure  identifications 

1.  Missions  (weapon  systems),  space  exploration  projects 

2.  Resource  requirement  breakouts 

3.  Time  periods 

C.  Discussion  of  Air  Force  force-costing  format 

D.  Interrelationships  of  individual  systems  or  projects  within 
a  force  structure 

1.  Performance  of  a  military  mission:  e.g.,  an  air  defense 
fighter  squadron  and  an  early  warning  radar  station 

2.  Resource  requirements  level 

a.  base  facilities 

b.  manpower  (common  special  skills  pool:  i.e.  ,  pilots) 

c.  equipment  development  and  procurcmei'.t 

E.  Necessity  of  treating  with  force  structure  considerations  to 
engage  in  incremental  costing  meaningfully. 

IV.  Effects  of  Differences  in  Time  Context  Upon  Resource  Projection 
A.  Individual  equipment 

1.  Long  range;  parametric  procedure 

2.  Short  term;  extension  of  cost  quantity  curves 
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TOOLS  OF  COSY  ESTIMATION 

Individual  Equipment  or  Operation  CKK-'a  (Cost-Estimating 
Relationships) 

A.  Definition:  expression  of  cost  as  a  function  of  physical 
characteristics,  performance,  and/or  operational  concept. 

3.  Uses 

1.  Projecting  a  major  element  jn  the  evaluation  of 
alternative,  future  weapon/space  systems. 

2.  Selection  of  an  optimum  configuration  during  preliminary 
design  (equipment). 

C.  Commonly  used  forms:  linear  multivariate,  exponential  or 
log- 1 inea r .  curvi linear. 

D.  Examples 

1.  Depot  maintenance  cost  as  a  function  of  aircraft  cost 
and  combat  speed. 

2.  Turbojet  engine  development  cost  as  a  function  of 
maximum  thrust  and  quantity  milestones. 

E.  Deriving  CER's— criteria  for  the  selection  of  explanatory 
variables. 

1.  logical  or  theoretical  relation  of  the  variable  to  cost. 

2.  Statistical  significance  of  the  variable's  contribution 
to  the  explanation  of  cost. 

3.  Independence  of  the  contribution  made  by  the  variable 
to  the  explanation  of  cost. 

F.  Limitations  of  CER's 

1.  Characteristically  (aerospace  industry)  small  sample 
sizes. 

2.  Extrapolating  a  new  equipment  whose  performance 
characteristics  exceed  those  of  most  or  ail  of  the 
cases  in  the  original  sample, 

a.  diverging  prediction  intervals 
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G.  Cost-Quantity  Relationships 

1  Relationship  of  cum  av  and  unit  cost  functions 

2.  Use  of  cum  av  and  unit  cost  curves  in  projecting 
individual  item  and  lot  average  costs. 

II.  Individual  Weapon  System  Requirement:  Identification  Displays 

A.  Matrix  of  GSE  costs  by  physical  location  and  type  of 
equipment. 

B.  Usefulness  of  such  cross  sectional  displays  in  checking 
the  completeness  of  a  system  estimate  and  measuring  changes 
in  the  estimate  with  changes  in  system  configuration. 

III.  Force  Structure  Distribution  Models 

A.  NASA  Manned  Space  Exploration  Model 

1.  Compiling  of  physical  requirements  by  like  items 
demanded  in  a  single  year 

2.  Application  of  CER's,  entering  throughputs 

3.  Application  of  time  lag  factors 

4.  Compiling  of  time-phased  financial  requirements  by 
individual  exploration  project 

B.  Interrelated  Resource  Requirements — Joint  Cost  Allocation 
Problem 

1.  Need  for  end  item  (mission)  identification  in  force 
structure  costing. 

2.  Multiple  use  resources 

a.  nonrecurring  requirements;  e.g,,  booster 
development,  launch  facility  construction 

b.  recurring  requirements;  e.g.,  tracking  network 
operations  ,  engine  procurement  cost  (cost-quantity 
effect) 

3.  Methods  of  allocation 

a.  proration  on  the  basis  of  the  proportion  of  the 
resource  consumed  by  user  projects 

b.  first  user 

c.  independent  project  status 

4.  Consumption  proration 

a.  advantages;  neatness 
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b.  disadvantages:  reallocation  of  resources  in  the 
case  of  each  force  structure  examined,  difficult 
to  distinguish  joint  product  cost  after  allocation 
has  been  made,  and  if  largo  may  bias  the  case 
agfeinst  a  given  project 

5.  First  user 

a.  advantages:  effect  on  the  using  projects  more 
easily  understood  and  more  clearly  shown. 

b.  disadvantage:  heavy  bias  against  one  project 

6.  Independent  project 

a.  advantages:  bias  of  arbitrary  allocation  removed, 
simple  to  identify  joint  product  cost 

b.  disadvantages:  format  complicated  by  additional 
element,  difficulty  in  evaluating  alternatives 
without  some  allocation  of  joint  cost 

TIME  PHASING  AND  DISCOUHTIKG 

I.  Time  Phasing 

A.  Importance  in  resource  analysis 

1.  Determination  of  economic  impact 

2.  Evaluation  of  inherited  assets 

B.  Financial  measurement 

1.  Expenditures 

a.  treasi  *y  disbursements 

2.  Program  requirements 

a.  obligational  authority  possessed  by  Government 
each  year 

II.  Discounting 

A.  Definition— the  application  of  soma  selected  rate  of  interest 
to  measure  the  differences  in  importance  ox  preference 
between  income  at  the  present  time  with  anticipated  income 
in  the  future. 
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B.  Time  preference 

1.  To  the  individual  or  firm-' -preference  for  present 
income  or  cost  savings  over  deferred  income  or  cost 
savings. 

2.  Not  so  clear  in  the  case  of  Government — which  is  not 
in  the  business  of  making  resources  grow  for  the 
future.  Main  Government  interest  in  maximizing  current  or 
near  future  capability  while  living  within  fixed  budgets. 

C.  Use  in  the  calculation  of  risk  or  uncertainty 

1.  Possible  application  to  Government  decisions  and  resource 
planning.  Vhile  perhaps  dangerous  for  resource 
estimation  where  future  costs  tend  to  exceed  early 
estimates,  may  be  useful  for  general  planning  purposes. 

D.  Present  value  discounting  method 

1.  Amount  of  money  deposited  at  interest  at  the  beginning 
of  a  system  life  and  drawn  on  for  all  needs,  would 
reduce  to  zero  at  the  end  of  system  lifetime. 

2.  Computations: 

PY  =  “  _ L_ 

t=l  (l+i)* 

where : 

PY  =  present  value, 
t  =  time  period, 

C  =  cost  in  time  period  t, 
i  =  interest  rate. 

E.  Interest  rate  controversy 

0  -  257.  (257.  when  used  as  an  uncertainty  adjustment) 
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UNCERTAINTY  AND  COST-ESTIMATING  ERROR 

I.  Requirements  Versus  Cost-Estimating  Uncertainty 

A.  Requirements  uncertainty  refers  to  variations  due  to  changes 
in  configuration  or  force  structure. 

B.  Cost-estimating  uncertainty  refers  to  variations  which  occur 
when  the  configuration  or  system  is  essentially  constant. 

II.  Requirements  Uncertainty 

A.  Number  of  empirical  studies  point  to  requirements  uncertaintj 
as  the  major  source  of  uncertainty  in  the  estimation  of 
aerospace  systems  and  force  structures. 

B.  Sources  of  requirements  uncertainty 

1.  Alteration  in  the  original  by  desired  performance 
characteristics  due  to  changes  in  the  overall  strategic 
picture. 

2.  Alterations  in  original  design  specifications  after 
discovering  they  will  not  provide  desired  performance 
charac  ter is  tics . 

3.  Alterations  in  originally  specified  IOC  dates. 

4.  Discovery  o:c  errors  of  omission  in  establishing 
requirements  for  some  part  of  the  system. 

C.  Requirements  uncertainty  basically  due  to  the  fact  that  cost 
estimates  are  prepared  for  a  fixed,  static  configuration, 
while  design  configuration  characteristically  undergo  fre¬ 
quent  and  substantial  change  during  their  development. 

D.  RAND  and  Harvard  Business  School  studies  of  the  variation 
in  cost  estimates  from  preliminary  design  through  delivery 
of  the  operational  article  have  found  variations  to  range 
as  high  as  a  factor  of  4  to  1  in  some  cases  and  to  average 
about  200  percent.  One  of  these  studies  suggested  a  20  to  30 
percent  factor  as  valid  for  cost  uncertainty  type  errors  alone 
This  20  to  30  percent  assumes  that  the  estimates  are  not 
political;  i.e.  ,  deliberately  misstated  in  any  way 
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III.  Cost -Estimating  Uncertainty 

A.  Sources  of  co3t-estimating  uncertainty 

1.  Errors  in  cost -estimating  relationships 

a.  normal  regression  theory  provides  an  estimate 
of  Y  as  a  function  of  X  within  calculable 
prediction  intervals--the  formal  statistical 
model  accepts  the  existence  of  error. 

2.  Errors  in  data  base 

a.  errors  of  measurement,  errors  in  the  observations 
from  which  the  relationship  had  been  derived 

3.  Extrapolation  errors 

a.  errors  in  estimates  of  Y  for  values  of  X  beyond 
those  subtended  in  the  data  base. 

4.  Price  level  changes 

a.  extrapolations  made  by  contractors  for  possible 
wage  rate  changes  and  material  price  changes 

b.  institutional  changes  in  the  industry — overhead 

5.  Errors  due  to  aggregation 

a.  differences  between  estimates  made  at  different 
levels  of  aggregation 

6.  Miscellaneous  errors  pertaining  to  equipment 

a.  subcontracting  structure 

b.  contractor  variation 

c.  changes  in  the  manufacturing  state  of  the  art 

d.  use  of  exotic  materials 

IV.  Treatment  of  Uncertainty  in  Cost  Analysis 

A.  Limited  usefulness  of  conventional  statistical  measures 
such  as  confidence  limits,  prediction  intervals 

1.  Saall  sample  sizes-^dif ficulty  of  establishing  independ¬ 
ence  among  the  explanatory  variables. 

2,  Not  applicable  to  key  requirement  uncertainty  problems. 

B.  Magic  formula  approach  to  the  downward  bias  of  aerospace 
industry  estimates, 

1.  More  useful  when  employed  for  a  large  number  of  cases-- 
may  not  work  in  preparing  a  particular  estimate. 
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2.  Up  to  present  such  extrapolations  are  prepared  from 
limited  size  samples  only. 

V.  Cost-Sensitivity  Analysis 

A.  Individual  aerospace  system  requirements:  the  examination 
of  how  cost  changes  as  key  characteristics  (including  botv 
hardware  design  and  operating  concept)  are  varied  over 
their  relevant  ranges. 

B.  Total  force  structure  requirements:  the  examination  of  how 
cost  varies  with  changes  in  the  configuration  characteristics 
of  individual  systems  in  a  force,  changes  in  force  size, 

and  force  mix. 

C.  Cost-estimating  uncertainty:  how  system  or  force  costs 
vary  due  to  uncertainties  in  cost-estimating  relationships , 
errors  in  basic  data,  extrapolation  error  and  the  like. 

D.  Uses  of  sensitivity  analysis 

1.  Examination  of  the  cost  implications  of  all  interesting 
system  and  force  possibilities. 

2.  Provides  range  of  cost  estimates  for  future  systems 
rather  than  individual  point  estimate. 

3.  Provides  relative  measure  of  the  sensitivity  or 
insensitivity  of  system  costs  to  variations  in  parti¬ 
cular  configuration  characteristics. 

E.  Sensitivity  analysis  examples 

1.  Missile  system  cost  versus  payload  weight  versus  ground 
environment  sitomation. 

a.  insensitivity  of  cost  to  payload  variation  because 
important  elements  of  cost — guidance ,  some  GSE— 

not  effected  by  missile  size.  Possibility  of  procuring 
higher  payload  missiles  at  relatively  minor  cost 
increments. 

b.  sensitivity  of  cost  to  ground  environment  automation 
and  the  diminution  of  system  personnel  requirements. 


2.  Nissile  system  cost  versus  reliability 

a.  significant  sensitivity  of  cost  to  both  mean  time 
to  failure  and  successful  launch  probability; 
justification  for  an  extensive  R&D  program  aimed 
at  fiuproving  guidance  reliability. 

3.  Recoverable  versus  conventional  booster  comparison 

a.  determination  of  the  level  of  demand  for  space 
transportation  which  justifies  the  development  and 
other  start-up  costs  of  a  new,  more  efficient 
vehicle. 

b.  effects  upon  this  cross-over  point  of  improvements 
in  the  currently  operational  booster;  i.e.  ,  either 
a  decrease  in  cost  or  Increase  in  payload  carrying 
capability. 

F.  Limitations  of  sensitivity  analysis 

1.  Presents  a  large  volume  of  difficult  ->-display  numbers 
to  a  system  analyst  who  really  wants  o»  *  figure  to 

run  with. 

2.  Provides  no  formal  measures  of  uncertatatv  statistical) 
and,  therefore,  no  probability  statements. 

3.  No  guarantee  that  any  given  sensitivity  analysis  has 
included  all  the  relevant  alternatives. 


INDIVIDUAL  WEAPON  SYSTEM  INPUTS 

OPERATIONAL  AND  ORGANIZATIONAL  OATA 
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INDIVIDUAL  SYSTEM  COST  CATEGORIES 

(MANNED  AIRCRAFT ) 
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AVIONICS 

FLIGHT  TEST  OPERATIONS 
FLIGHT  TEST  SUPPORT 


INDIVIDUAL  SYSTEM  COST  CATCGORICS  fCONT) 

(MAN VCD  AIRCRAFT) 
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MY  I  DUAL  SYSTEM  COST  CATEGORIES  (CQNT) 

(MANNED  AIRCRAFT) 

•  ANNUAL  OPERATION  COSTS 

•  FACILITIES  B*  M 


•  ANNUAL  TRAVEL 

•  ANNUAL  TCANSPOCTATION 

•  ANNUAL  SEBVICES 
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